Online Exam Delivery Tips: The Online Open-book Examination

Following on from Mathew Hillier’s advice on exam alternatives and Sean Brawley’s piece on online oral examinations, Erik Nielsen looks at another online examination option.

The Online Open-book Examination: Opportunity and Risk in the Time of Corona

An Online Open-book Examination (OObE) provides a useful option for unit convenors looking to replace invigilated exams within their assessment suites. OObEs can act as an appropriate analogue for an invigilated exam in the current circumstances, and have advantages over invigilated exams in terms of what they can assess. The big risk in terms of using OObEs is the perception that they are easy marks for cheating in the form of collusion, plagiarism or contract cheating. Physical invigilation provides an almost inviolable assurance regime for academic integrity that is hard to replicate, but there are ways to build academic integrity into the design of OObEs without the financial and technological cost of invigilation. The online environment can be used to the advantage of unit convenors to test skills such as information literacy that are not traditionally assessed in exams.

The iLearn team has developed extensive advice about developing a quiz through iLearn that can be utilised as an Online Open-book Exam. The following sections provide advice about how to use the iLearn functionality to develop an OObE that assesses students effectively and has academic integrity features built into its design.

Ensure that the Exam has a Strict Time Limit

A time limit puts pressure on a student to perform in an analogous way to an invigilated exam. Requiring a student to answer questions within a tight time frame means that their opportunity to consult texts and any other materials is limited. Students will have to know enough about course material to answer questions well enough to perform in the exam. An unprepared student will not be able to rifle through their papers or the internet to come up with the required answer often enough in the case of Multiple Choice Questions, or give themselves enough time to complete short or long responses.

Use a Question Bank

A question bank can ensure that students can’t be sure exactly what questions they will answer. Question banks are usually most necessary for exams run in multiple years for the same unit, but can prevent collusion if students don’t complete the exam at exactly the same time. See ‘Use randomized quiz questions’ in the iLearn advice to add this function to your Online Exam.

Test Higher Order Skills Rather than Facts

OObEs that are focused on testing a student’s ability to recall facts are more open to breaches of academic integrity than exams that test higher order skills. Higher order skills can be tested in several general ways (there are also doubtless discipline-specific ways that are applicable):

Application of concepts encountered during the unit

Williams and Wong (2009, p. 229) describe a task where ‘a contemporary real-world problem [is] brought to life through the use of hyperlinks to websites and streaming media that serve to enhance the authenticity of the problem.’ Students apply the skills and techniques they have learnt through the unit to solve the problem set. Students are given strict instructions including a ‘Guide to the Task’ to ensure they don’t suffer writers block or misinterpret the task. Students are also required to refer specifically to materials that they have encountered throughout the unit. Requiring students to engage with unit content in their answers severely limits the opportunity to collude or contract cheat. The task becomes one of authentic assessment.

Embedding information literacy into the task

Information Literacy skills, including searching, finding, evaluating and using information, are not usually able to be assessed in an exam setting, but can be tested during an Online Open-book Exam. It’s important that a realistic task is set within the confines of the exam. The task outlined by Williams and Wong represents a way of examining elements of information literacy, with a focus on evaluating and using information.

Reflection on experiences obtained during the unit

A question can be written that requires students to reflect upon a significant experience they had during the unit. This is an excellent way to prevent academic integrity breaches, as it would be practically impossible for a student to plagiarise a response. This option is particularly well suited to a unit that has a scaffolded task, an extensive individual or group project or a placement.

Like all assessment suites, OObEs are best utilised in a constructively aligned environment that is compromised by these emergency conditions. The short timeframe available to make required changes may make it impossible to utilise all the functionalities of an OObE this time around, but thinking about these issues now might allow future offerings to benefit from this approach.

Further Reading

Kevin Casey, K. Michael Casey and Ken Griffin, ‘Online Learning and Competitiveness: Incorporating Teaching Strategies and Software That Encourage Ethical Behavior’, Competition Forum, 17 (2), 2019, pp. 267-74.

Jeffrey R. Stowell, ‘Online Open-Book Testing in Face-to-Face Classes’, Scholarship of Teaching and Learning in Psychology, 1 (1), 2015, pp. 7-13.

Kim Welch, ‘Alternatives to face-to-face tests’, 14 March 2020, https://blogs.chapman.edu/academics/2020/03/14/remote-assessments/ accessed 7 April 2020. A brief post by a learning designer from Chapman University that lists alternatives to face-to-face tests.

Tom Wielicki, ‘Statistical Measures of Integrity in Online Testing: Empirical Study’, International Conference e-Learning, 2016, pp. 169-173.

Jeremy B. Williams and Amy Wong, ‘The efficacy of final examinations: A comparative study of closed-book, invigilated exams and open-book, open-web exams’, British Journal of Educational Technology, 40(2), 2009, pp. 227-36.